Overview

The IEC High-Performance Computing (HPC) environment is designed to tackle complex computational problems by harnessing massive parallel processing power. These HPC platforms integrate advanced hardware components, including multi-core processors, GPUs, high-speed interconnects, and large-scale storage systems. Coupled with sophisticated software frameworks, they enable efficient and scalable execution of compute-intensive tasks. This capability is vital across various fields, such as scientific research, engineering, and data analysis.

Key Elements and Features

1. Computing Resources
   IEC implements HPC on Kubernetes, which efficiently manages computer resources and automatically scales them up or down as needed. This allows HPC applications to be packaged into containers, making them flexible and adaptable for various jobs.
2. Parallel Processing
   IEC enables robust batch job processing, allowing users to submit multiple computational tasks that run automatically without requiring direct interaction. This ensures efficient resource utilization and high throughput for large-scale workloads.
3. Advanced Storage Solutions
   IEC provides high-performance storage solutions to accommodate the data needs of HPC. The external storage server (GS/GSx) is optimized for high throughput and low latency, serving as the central hub for:
   • Applications
   • User data
   • Job results
   • Logs
   This centralized system is essential for supporting data-intensive computing tasks.
4. Comprehensive Software Stack
   IEC HPC supports a wide range of applications, libraries and job types, enhancing versatility:
   • Applications: The platform supports variety applications across different domains, including:
     • Life Sciences: GATK (genomic analysis), RELION (electron microscopy), Gromacs (molecular dynamics).[K1]
     • Computational Science: Blast (genomic comparisons), GAMESS (quantum chemistry), MATLAB (numerical computing).
     • Earth Sciences: WRF (weather research), OpenFOAM (fluid dynamics).
     • Natural Sciences: ParaView (visualization), HPC Desktop (local computations).
   • Libraries:
     • Singularity: A container platform for running applications in isolated environments.
     • CUDA: NVIDIA's platform for GPU programming.
     • PyTorch: A deep learning framework that simplifies GPU usage.
     • OpenMPI: A library for message passing in parallel computing environments.
   • Job Types: Users can run jobs in various modes:
     • Interactive Jobs: For real-time computation and debugging.
     • Batch Jobs: For scheduled, unattended execution.
     • Desktop Jobs: Lightweight computations interfacing with HPC.
5. Seamless Application Access
   Users can run applications through multiple methods:
   • Marketplace Access: A centralized hub for deploying pre-built, optimized HPC applications.
   • Custom Scripts: Users can submit tailored scripts in Python for personalized workflows.
   • Spack: Simplifies the installation and management of software dependencies.

Conclusion

IEC HPC empowers users to efficiently run a variety of applications, utilize robust storage solutions, and manage different job types. By harnessing these capabilities, users can drive innovation and research across diverse disciplines, making HPC an invaluable resource in today’s data-driven world.

To further enhance the user experience with IEC HPC, the following guides provide essential steps for getting started:

Getting Started with IEC HPC

• Explore the Marketplace: Browse the wide selection of pre-configured HPC applications and select the one best suited for your needs.
• Accessing the File Explorer: Understand that the user's main work and commonly used data are located in the IEC system. It is the IEC main storage system and the default file system that users enter after logging into the node.
• Uploading Data: Prepare and import your data of jobs or projects to your home directory for further analysis and processing.
• Submitting a Job from Marketplace: Learn how to easily submit your computational tasks, whether it's a complex simulation or a large-scale data analysis.

Managing HPC Jobs

• Monitoring Jobs: Stay up-to-date on the status of your submitted jobs, tracking their execution in real-time.
• Managing a job: To ensure resource efficiency, you can stop/ delete a job during the job execution or the job is failed.
• Managing Job Results: After the job is completed, organize and manage the output files generated. This may involve renaming files, moving them to appropriate directories, or archiving them for long-term storage.

Advancing Your HPC Workflows

• Collaborate with others: Leverage the shared storage and project user management features to enable teamwork and collaboration on HPC-powered projects.
• Run Customize Job Scripts: Discover how to create and submit tailored scripts using programming languages like C and Bash to automate and streamline your workflows effectively.
• Troubleshoot and Debug: Access detailed logs and job information to identify and resolve any issues that may arise during job execution.

By following these comprehensive guides, users are able to harness the full potential of IEC HPC, driving their research and innovation forward in this data-centric era.